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Automation methodology for 3 D image metrology systems 
TECHNICAL FIELD 

The present invention relates to 3D-image metrology with 
photogrammetric methods . 

The invention is applied to 3D Image Metrology but can also 
be applied to other electro-optical systems. 3D Image 
Metrology is applicable to all areas where 3D geometric and 
other information is to be extracted from imagery, imagery 
being acquired through standard imaging systems or other 
devices such as imaging radar or sonar. Typical application 
areas are dimentional measurements to be performed in the 
industrial process reaching from design, development, 
production to quality control, in engineering but even in 
areas such as art and medicine . 

BACKGROUND OF THE INVENTION 

The background of the invention is methodologies for the 
automation of so far manual tasks in 3D image metrology 
systems . 

For exemple, robots today are programmed off line which 
means a teaching of robots using a computer simulation 
program. Due to technical limits in the mechanical 
movements of a robot arm, there is a difference between the 
programmed position of a robot arm and its real position. 
This differecne can be measured with the robot calibration 
system and correction polynoms for the different ankles can 
be derived. Using such a correction for the simulation, the 
accuracy of the real robot movement can be increased. Three 
or four CCD-cameras are mounted in the vinicity of the 
robot to observe the measurement volume. The cameras detect 
auto- recognizable targets mounted on and in a specific 
relation to the tool center point. According to these 
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detections a calibration process is activated and 
corrections for the simulation is established. 

Targets were designed to faciliate automatic detection and 
5 identification. For exemple, the targets could be designed 
as a white circle with a concentric ring on a black base . 
Around the concentric ring there is another concentric ring 
as a label area in which a ten bit target identification is 
coded. The central circle has a black dot in the middle to 
10 faciliate manual measurement. Black and white are colors 
chosen to get the maximum contrast. 

Therefore, part of the invention, i.e. auto- recognizable 
targets, is already used. 

A large number of attemts to arrive at this goal of 
automation have been undertaken. All attemts so far have 
only achieved partial results and/or are not suited for 
widespread use . 

One approach of automatically identifying individual 
targets was relying on different forms and shapes of the 
targets . Targets which were made up of various geometric 
shapes such as circles, rectangles and crosses were used to 
differentiate between 2D patterns of targets. This approach 
need relatively large targets and does not allow for a 
large number of different patterns. 

Another approach used targets with an additional linear 
3 0 pattern area in the vicinity of circular targets. The 

pattern was similar to typical bar codes. The code was such 
not integrated with the target per se and needed relatively 
much space. These targets were so called "coded" targets 
and were developed by several groups at that time. 

35 

Another approach with "coded" targets uses a bit pattern 
around the target. The pattern was a binary bit pattern 
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which was designed to be independent of the rotation of the 
taget. Two major disadvantages were common to these 
targets . They had too small distance between target and 
code when used with CCD-cameras and they could only use the 
5 labels provided by the code, i.e. typically numbers like 
1,2,3 but no alphanumeric labels. The insufficient amount 
of space between target and code degraded the measurement 
accuracy of the target. Groups with several hundred targets 
were discernible with that approach. All codes provided 
10 only to use the same label for the target as the label 
which was encoded in the labeling area of the target. 

Furthermore, groups of targets were used by some systems to 
identify one particular group and through that group 
15 individual targets of the group. These techniques rely on 
the particular geometric relationship of all targets in a 
group which can either be identified in an image and/or in 
3D space . 

20 Other approaches to identify an element was to use groups 

of targets which lie in a particular geometric relationship 
to eachother and which discern the group from other targets 
within an image or even within the 3D space. Disadvantages 
of these techniques ly in the fact that one always needs 

25 several targets and that the technique does not allow for a 
large number of different groups with resonable economic 
effort and space requirements. 

The above approches were furthermore combined with the use 
30 of colours in order to increase the number of targets which 
are discernible. 

Another aspect of automation is the establishment of 
correspondence of targets which are not discernible through 
35 their particular individual characteristics such as a 
label, a colour, a form. Two basic techniques were 
developed to overcome this problem. One uses the geometric 
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relation between images from which geometric conditions can 
be derived, i.e. the so called epipolar line. Using this 
technique the correspondence between large numbers of 
targets could be established automatically. A similar 
approach where the geometric relation between images and 
images and the object space is used is the Multiphoto 
Geometrically Constrained Matching. In another approach the 
3D relation of points was used directly. Both techniques 
only adressed part of the problem as they required that the 
exterior orientation and/or the relative orientation of 
images had to be known a priori . 

The establishment of the correspondence of homologous 
points in images can be performed by the use of the 
epipolar geometry, as well as using the 3D relation of 
points . 

These approaches of the known solutions to automatically 
identify individual targets were rather limited and shows 
disadvantages in the following ways: 

1. The targets could not be measured with sufficient 
accuracy (not for all techniques) . 

2. Only numeric labels were associated with the targets 
although in many applications alphanumeric labels are 
required. 

3 . Groups of targets require much more space than a target 
with an associated label area. 

4 . Groups of targets are not sufficiently discernible from 
other targets in applications with large numbers of 
targets . 

The automatically indentif lable targets have priviously 
been designed with film based 3D image metrology systems in 
mind. They did not take care of the imaging characteristics 
of CCD- sensor or other digital imaging systems. 
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The establishment of the correspondence did not use auto- 
identifiable targets to establish the initial orientation 
of the images. With respect to a complete system they did 
not allow to: 

Automatically identify the object. 

1. Provide for a mechanism to automatically load all the 
relevant data for a particular application. 

2. Provide for highest measurement accuracy, immunity to 
occlusions, and alphanumeric labels. 

3. Provide for automatic identification of adapters 
independent of knowledge on their spatial location. 

Overall, the existing methods so far were only addressing 
part of the problem. 

OBJECT OF THE INVENTION 

:o 



The object of the invention, in parts or total, is t< 
provide to partial and/or complete automation for 

20 dimentional measurements with image metrology. Most 
important are systems where 3D image metrology is to 
perform fully automatically, such as in automated 
production systems where robots or other manufacturing 
systems are to be guided, e.g. for machining, fitting, 

25 moving. 

An other object of the invention is to provide for a higher 
degree or even total automation in 3D image metrology 
and/or other applications where such systems are used. 



SUMMARY OF THE INVENTION 

The invention applies to the following subtasks : 



35 



Finding targets for the purpose of automatically 
determining the exterior orientation of images. 
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Finding targets in several images for the purpose of 
establishing homologous point in these images. 

Finding targets from different views in an electro- 
optical system in order to establish their relation. 

Identifying targets which serve as references for the 
definition of a coordinate system. 

Identifying measurement adapters in order to identify 
the particular adapter and its geometric and/or other 
characteristics . 

Examples for such adapters are: 

- Individual targets for which some offsets and other 
geometric parameters are known. 

- Two or more targets forming a straight line with 
another spatial location which they refer to. 

- Three or more targets which are in a spatial relation 
to one or more other spatial locations. 

Examples thereof are: 

- Adapters consisting of three or more visible 
targets and three or more half spheres which lie on 
an object surface. The visible targets and the 
other locations are in a known relative spatial 
relation . 

- Adapters consisting of three or more visible 
targets which are in relation to one or more 
locations. The visible targets and the other 
locations are in a known relative spatial relation. 
Physical examples are the DigiPen, which is a 
touchprobe of the Imetrics systems, or a spindle 
and the targets on the spindle holder in a TI 2 
system . 

- LUT 

The invention will be part of a future improvement of a 3D 
image metrology system. It is applicable to other systems. 



WO 97/31336 



PCT/IB97/00236 



7 

It will also be used in an industrial manufacturing system. 
DESCRIPTION OF THE INVENTION 

5 The invention uses the following steps in the current 

implementation either on an individual basis or as in any 
combination of the individual techniques . 

1. Measurement Accuracy: The design of the auto- 

10 identifiable targets is such that the distance between 

the central target and the label area, which is used to 
identify the target, i.e. the area with a binary code or 
. other code, is sufficiently large in order not to 
degrade the accuracy with which the central target can 

15 be located. This distance was determined to be at lease 

equal to the diameter of the central target when the 
central target was designed to provide for an optimum 
measurement accuracy, i.e. to be imaged on typically 6 
pixels in diameter. 

20 

2. Immunity to partial occlusions of label area: In many 
applications, part of the area containing the label can 
be occluded from other objects lying between the imaging 
system and the label area. Techniques using a redundancy 

25 in the code, additional areas such as a border around 

the label, and a -priori knowledge on which auto- 
identifiable targets were used, can be used individually 
or in combinations to circumvent this problem. 

30 3. Labeling Freedom: To overcome the problem of having to 
apply particular targets on particular points which one 
would like to label differently, a translation between 
the label of the target and its physical label is used. 
Such an auto- identifiable target with the label "11" can 

35 be used as a target with a label "A" . This can for 

example be implemented using a look up table where a 
"user" label is connected to each physical label of a 
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target. This look up table can be user definable or 
fixed. Furthermore a double look up table can be used, 
i.e. a table where the label provided by the label area 
is first translated into another label and later is 
finally related to a label to be used in the further 
processing. For example, the label on the target 
contains the label "11" • On the target, the name "A" can 
be used to identify it to a person, but in the 
particular application the target is to be used as 
target M X" . This helps to reduce the number of auto- 
identifiable targets physically required in order to 
fullfill a large number of different applications. 

4. Identification of the Measurement Object: Here, one or 
more labels of auto- identifiable targets are used to 
identify a particular object. For example, the hood of 
model A of a car will have auto- identifiable targets 
with label f, K" and the hood of model B of a car will 
have auto- identifiable targets with label "Y" . One or 
more labels can be used. Based on this information, data 
related to the particular object can be automatically 
selected. Use of such an object identification as a safe 
guard (for example in manufacturing, but also when 
performing quality inspection) and/or to load data 
related to the particular object. 

5. Default Project Strategy: A methodology used in 
connection with the auto- identifiable targets where the 
data pertaining to a particular task (e.g. coordinates 
of reference locations, compensation for adapters, 
measurements to be performed) is collected in a "default 
project" . This can be a collection of files containing 
the pertinent data or a particular access to a data base 
or some other technique to store and access the relevant 
data. 



6. Automated Orientation: 
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A procedure where in the setup of a measurement system 
auto- identifiable targets are used to automatically 
compute the exterior orientation and/or relative 
orientation of one or more cameras/images. 

A procedure where during measurements auto-identifiable 
targets are used to automatically compute the exterior 
orientation of one or more cameras/ images in order to 
account for eventual changes in the relation between the 
object (s) on which the auto- identifiable targets are 
fixed and the camera (s) and/or to perform other 
measurements . 



Any procedure where three or more auto- identifiable 
15 targets are used to establish a particular coordinate 

system. 

In all these procedures, the 3D coordinates of at least 
three auto- identifiable targets must be known. 

20 

7. Automated Measurement: 

Procedures where auto- identifiable targets are used to 
establish the geometric relation of images with respect 
to a coordinate system related to the auto- identifiable 

2 5 targets (exterior orientation) or to some arbitrary 

coordinate system, for example related to one or more of 
the cameras/ images (relative orientation) in order to be 
able to establish the correspondence between standard 
targets in two or more images. 

30 

Determining the exterior orientation of individual 
images using for at least three auto- identif iable 
targets of which the 3D spatial coordinates in some 3D 
coordinate system are known, either prior to start of 

3 5 measurements or which were determined in the course of 

the measurements . 
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Determining homologous points in images. This can for 
example be used to compute the relative orientation of 
these images . 

5 Combination of auto- identifiable targets and standard 

targets. This technique allows to establish the spatial 
geometric relation of the images, i.e. either their 
exterior orientations or their relative orientations, 
through auto- identifiable targets and eventual other 

10 targets which were already located in the images but not 

identified (i.e. the relation between a particular 
target in an image and its origin in object space and/or 
its corresponding image in another image is not known) , 
and to establish the correspondence between homologous 

15 targets imaged in different images using the geometric 

relation of these images (i.e. their exterior 
orientations or their relative orientations) and/or the 
3D geometry of the targets to be identified which 
becomes computable using the geometric relation of the 

2 0 images in which they were imaged. 

Targets can be retro-reflective targets but also any other 
feature of an object which can be located in an image or a 
group of images . 

25 

8. Scale Bar: The use of one or more auto- identifiable 

targets to identify a particular scale bar in order to 
obtain its calibrated distance or distances (if more 
than two points are located on the scale bar) . The 
30 following elements are part of the technique: 

1) Identify the particular scale bar 

2) Help in the search for others targets on the scale bar 

3) Use several known distances on one scale bar for the 
purpose of improved accuracy and reliability. 

35 They can be applied individually or in combinations. 



9. Particular adapters or object are identified by using 
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either one or more of the labels. Using a single or a 
group of adapters is then in turn used. 

One or more labels are used to identify a particular 
object. Such an object can be: 

- An individal point with or without some given 
characteristics such as XYZ-coordinates for the 
purpose of orientation. Using the label, the 3D 
spatial coordinates of a point can be taken from a 
data base in order to compute exterior orientations, 
to compute spatial position of an object with respect 
to some coordinate system through a spatial similarit 
transformation . 

- An individual label serving to identify an adapter 
with its characteristics. These can for example be th 
offset of a button target, the vector distances for 
reductions of double vector targets or hidden point 
bars . 

- A multi-point adapter where the label is used to find 
the geometric relation between visible targets and 
another point or points which are in particular 
geometric relation to the targets. 

- A particular scale bar. 

- A particular touch probe or other device such as a 
spindle or other manufacturing device and other 
objects which are to be identified. 

Adapters are any technique where one or more auto- 
identifiable targets on any sort of adapter is to: 

- Identify the particular adapter to derive geometric 
parameters of the adapter. 

- Identify the particular adapter to retrieve its 
spatial coordinates from a data base or other storage 
mechanism . 

- Identify the particular adapter to control the 
software such that it performs a particular 
measurement sequence or other process . 
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Examples: 

a) The system finds a double vector target and 
computes automatically the mechanical point 
related to the two optical targets. 
5 b) The system finds a particulat target group and 

commands a robot to go and grab the piece. 

The following first defines a number of adapters and 
then defines the use of the auto- identifiable targets 
10 with these adapters. 

Single Point Adapter: A "Single Point Adapter" is a 
device consisting of one optically visible target, e.g. 
a retro-reflective target or LED, and one or more 
15 mechanical points, e.g. the location of a shank, the 

center of a sphere where the relation of the optical and 
mechanical targets are known and usually fixed. 

Typical examples are : 
20 - A "Button Target" which consists of one optical target 

sitting on a mechanical piece with a shank adapted to 
tooling holes . The target is in the centre of the 
shank axis but at a certain distance from a mechanical 
interface . 

25 - Target integrated into a partial sphere as for example 

targets integrated into a Taylor Hobson sphere. 

The auto- identifiable target is used to identify an 
individual "Single Point Adapter" and to extract some 

3 0 generally geometrical characteristics from a data base. 

Such information can be the offset of the optical target 
and the one or more mechanical points. This allows to 
use adapters with various offsets in one measurement 
cask without requiring the user to identify the adapter 

35 and without relying on applying one particular adapter 

with a particular offset to specific locations. 
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The auto-identifiable target is used to find the 3D 
reference coordinates of the "Single Point Adapter", 
i.e. either of the optical target or the reference 
location . 

A "Two Point Adapter" is a device consisting of two or 
more optically visible targets and one or more 
mechanical or virtual points. The differentiation 
between "Two Point Adapters" and "Multi-Point Adapters" 
is the fact that the first one does not allow to 
establish a 3D relation between the optically visible 
targets and the mechanical or virtual points without 
additional knowledge on the spatial location of the "Two 
Point Adapter". In geometric terms, the points of the 
"Two Points Adapter" cannot be used to determine a 3D 
spatial similarity transformation with six parameters. 
Usually the point (s) to be computed using a "Two Point 
Adapter" are on a 3D line with two optical targets. 

Typical examples of "Two Point Adapters" are: 
"Double Vector Targets". This adapter consists typically 
of two optically visible targets. The mechanical or 
virtual point of interest is on a spatial line with the 
two optically visible targets and its (spatial) distance 
from one or both targets is known or it may be in the 
geometric centre of the two optically visible targets. 

One or more auto- identifiable targets are used to 
identify a particular "Two Point Adapter" in order to: 

- Identify the adapter and obtain its geometric 
parameters and/or the naming convention for the one or 
more points to be computed. 

- Identify the adapter to retrieve the spatial 
coordinates of one or more optical targets and/cr one 
or more reference locations of the adapter. 

- Identify the adapter to control particular software 
functions or to indicate certain actions by the 
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system. 
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A "Multi- Point Adapter" is a device consisting of three 
or more optically visible targets which are in a 
geometric relation to an object and/or one or more 
mechanical or virtual points. The "Multi-Point Adapter" 
can for example be used to determine the spatial 
position and orientation of the adapter or an object 
physically connected to the adapter. Furthermore, it can 
be used to derive the spatial location of one or more 
points on the adapter or another object physically 
connected with the adapter. 

Examples of "Multi-Point Adapters" are: 

- V-Plates: A device consisting of three optical targets 
and three half spheres. 

- Robot Calibration Fixtures: Devices fixed to robots 
for the purpose of calibrating robots. 

- Touch Probes such as the DigiPen and other devices 
consisting of three or more optical targets connected 
to a CMM touch probe for the purpose of determining 
the 3D spatial coordinates of the probe. 

- Tooling Fixtures: Any device on a tool which serves to 
position a piece in a manufacturing production 
process. Adapters can be built to position tooling 
fixtures . 

- Tool holders or Tools (drills, grippers, cutters and 
other end effectors) : These can be on manufacturing 
systems such as robots or robot arms, CNC machines, 
but also on hand-held devices. 

The use of auto- identifiable target (s) for: 

- Identifying the particular adapter to derieve its 
geometric characteristics and/or to retrieve its 
spatial reference coordinates. 

- Identifying the particular adapter to initiate 
particular actions or computations to be performed 
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such as to automatically compute the invisible 
targets . 

Notes : 

5 - Targets to be mounted in tooling holes or other 

places with a target that can automatically be 
identified, labeled and measured by a computer 
program . 

Automatically measuring targets for the purpose of 
10 determining the orientation of one or more cameras 

automatically for the purpose of orienting cameras 
Update orientation. 

Tool with three or four targets to automatically 
orient cameras . 
15 - Devices with three or more points that are 

automatically identifiable and are used to derive 

1. the position and/or attitude of the device that 
they are attached to in space 

2. the position of one or more other points 

20 physically connected with these points in space. 

Tool to perform an automatic orientation and to 
automatically compute approximations for object points. 

25 Method where points are labeled by indicating their 

label in one image only. 

Coded targets to position other devices such as robots, 
CMMs , CNC machines . 

30 
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Claims : 

1. Method of 3D image metrology using auto-identifiable 
targets, characterized by determining the exterior 

5 orientation of images comprising targets and/or pattern or 
groups of targets. 

2. A method as claimed in claim 1, characterized by finding 
auto- identifiable targets in several images and 

10 establishing homologous points in these images. 

3. A method as claimed in any of the claims 1-2, 
characterized by finding auto- identifiable targets from 
different views and establishing their relative relation. 

15 

4. A method as claimed in any of the claims 1-3, 
characterized by identifying auto- identifiable targets 
which are defined as a reference for the definition of a 
coordinate system . 

20 

5. A method as claimed in any of the claims 1-4, 
characterized by identifying measurement adapters for 
identification of a particular adapter with its 
characteristics . 

25 

6. A method as claimed in any of the claims 1-5, comprising 
a central target and a label area characterized by applying 
the label area at distance from the central target beeing 
sufficiently large in order not to degrade the accuracy 

3 0 with which the central target can be located. 

7. A method as claimed in claim 6, characterized in that 
the distance is beeing set at a value least equal to the 
diameter of the central target when designed for an optimum 

3 5 measurement accuracy. 



8. A method as claimed in claim 7, characterized in that 
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the distance is determined as a result of the central 
target to be imaged on typically 6 pixels in diameter. 

9. A method as claimed in any of the claims 1-8, 

5 characterized by establishing immunity to partial 

occlusions of label area, for exemple techniques using a 
redundancy in the code, additional areas such as a border 
around the label, or a-priori knowledge on which auto- 
identifiable targets, can be used individually or in 
10 combinations. 

10. A method as claimed in any of the claims 1-9, 
characterized by that a translation between the label of 
the target and its physical label is made in order to 

15 establish a labeling freedom to apply particular targets on 
particular points which one would like to label 
differently. 

11. A method as claimed in any of the claims 1-10, 
2 0 characterized by using one or more labels of auto- 
identifiable targets to identify a particular object. 

12. A method as claimed in any of the claims 1-11, 
characterized in that data, e.g. coordinates of reference 

25 locations, compensation for adapters, measurements to be 

performed, pertaining to a particular task, is collected in 
a default project. 

13. A method as claimed in any of the claims 1-12, were 
30 the 3D coordinates of at least tree auto- identifiable 

targets is known, characterized by establishing an 
automated orientation by a procedure where in the setup of 
a measurement system auto- identifiable targets are used to 
automatically compute the exterior orientation and/or 
35 relative orientation of one or more cameras/images. 



14. A method as claimed in any of the claims 1-13, 
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characterized by establishing an automated orientation by a 
procedure where during measurements auto- identifiable 
targets are used to automatically compute the exterior 
orientation of one or more cameras/images in order to 
5 account for eventual changes in the relation between the 

object (s) on which the auto-identifiable targets are fixed 
and the camera (s) and/or to perform other measurements. 

15. A method as claimed in any of the claims 1-14, 

10 characterized by establishing an automated orientation by a 
procedure where three or more auto- identifiable targets are 
used to establish a particular coordinate system. 

16. A method as claimed in ainy of the claims 1-15, 

15 characterized in that an automated measurement is achieved 
by procedures where auto- identifiable targets are used to 
establish the geometric relation of images with respect to 
a coordinate system related to the auto- identifiable 
targets or to some arbitrary coordinate system, for example 

20 related to one or more of the cameras /images in order to be 
able to establish the correspondence between standard 
targets in two or more images. 

17. A method as claimed in any of the claims 1-16, 

25 characterized in that an automated measurement is achieved 
by determining the exterior orientation of individual 
images using at least tree auto-identifiable targets of 
which the 3D spatial coordinates in some 3D coordinate 
system are known, either prior to start of measurements or 

30 which were determined in the course of the measurements. 

18. A method as claimed in any of the claims 1-17, 
characterized in that an automated measurement is achieved 
by determining homologous points in images. 

35 

19. A method as claimed in any of the claims 1-18, 
characterized in that an automated measurement is achieved 
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by establishing the spatial geometric relation of the 
images, through auto- identifiable targets and eventual 
other targets which were already located in the images but 
not identified and to establish the correspondence between 
5 homologous targets imaged in different images using the 

geometric relation of these images and/or the 3D geometry 
of the targets to be identified which becomes computable 
using the geometric relation of the images in which they 
were imaged . 

10 

20. A method as claimed in any of the claims 1-19, 
characterized by identifing a particular scale bar in order 
to obtain its calibrated distance or distances. 

15 21. A method as claimed in any of the claims 1-20, 

characterized by applying one or more auto- identifiable 
targets on any sort of adapter. 

22. A method as claimed in any of the claims 1-21, 

20 characterized by identifying the particular adapter to 
derive geometric parameters of the adapter. 

23. A method as claimed in any of the claims 1-22, 
characterized by identifying the particular adapter to 

25 retrieve its spatial coordinates from a data base or other 
storage mechanism . 

24. A method as claimed in any of the claims 1-23, 
characterized by identifying the particular adapter to 

30 control the software such that it performs a particular 
measurement sequence or some other process. 

25. A device using 3D image metrology auto- identifiable 
targets, characterized by a determination apparatus for the 

35 exterior orientation of images comprising targets and/or 
pattern or groups of targets. 
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26. A device as claimed in claim 25, characterized by 
comprising any features or combination of features claimed 
in any of the claim 1-24. 
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